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Asymptotically  Optimal  Designs  for  Sane 
Time  Series  Models 

By  Randall  L.  Eubank,  Patricia  L.  Smith  and  Philip  W.  Smith 
Southern  Methodist  University  and  Old  Dominion  University 

Short  title:  Asymptotically  Optimal  Designs 

Summary.  The  limiting  behaviour  (as  the  sample  size  increases) 
of  the  BLUE  of  the  regression  coefficients  is  investigated  in  the 
case  that  derivative  information  is  not  available.  Using  the 
results  of  Barrow  and  Smith  (1979)  on  the  asymptotic  properties 
of  optimal  quadrature  formulas  several  results  obtained  by  Eubank, 
Smith  and  Smith  (1981)  are  extended  to  a  multiparameter  setting 
and  a  wider  class  of  processes  which  includes  multiple  integrals 
of  Brownian  motion.  The  asymptotic  behaviour  of  the  variance  of 
the  BLUE  is  characterized  in  terms  of  the  density  defining  the 
designs  and  densities  which  generate  asymptotically  optimal  design 
sequences  are  provided  for  several  optimality  criteria. 

1.  INTRODUCTION  .  Consider  the  linear  regression  model  in 

which  a  stochastic  process,  Y,  is  observed  having  the  form 
J 

(1.1)  Y (t)  -  l  8  f  (t)  +  X(t),  t  e  (0,1)  , 

j-1  3  ■’ 

where  8*»(B  ,...,8  )  is  a  vector  of  unknown  parameters,  the  f  are 
l  J  j 

known  regression  functions  and  X(*>  is  a  zero  mean  process  with 
known  covariance  kernel  R.  The  X  process  is  assumed  to  admit  k  -  1 
quadratic  mean  derivatives  at  each  point  t  c  (0,1). 
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If  the  Y  process  is  sampled  over  all  of  [0,1],  the 
linear  estimation  of  8  may  be  accomplished  through  the  use  of  the 
reproducing  kernel  Hilbert  space  (RKHS)  techniques  developed  by 
Parzen  (1961a,  1961b) .  We  denote  this  estimator  by  6  and  its 
corresponding  variance-covariance  matrix  by  A  \  When  observations 
are  taken  at  only  a  finite  number  of  distinct  design  points  on  [0,1] 
the  best  linear  unbiased  estimator  (BLUE)  of  6  can  be  constructed 
through  the  use  of  generalized  least  squares.  Various  aspects  of 
the  problem  of  optimal  design  selection  for  the  BLUE  have  been 
addressed  by  laacks  and  Ylvisaker  (1966,  1968,  1970) ,  Wahba  (1971, 

1974),  and  Eubank,  Smith,  and  Smith  (1981). 

Denote  the  set  of  possible  n+1  point  designs  for  model  (1.1)  by 
U.2)  Dn  :  -  {(to,  tx . tn)|  0  -  tQ  <  tx  <  ...  <  tn  -  1), 

where  means  "is  defined  as".  Also  for  T  cD^  let  8T  represent 

the  BLUE  of  8  based  on  the  observation  set  Y  =  (Y(t) It  e  t)  with 

T 

corresponding  variance~covariance  matrix  denoted  by  A^1.  When  k  =  1,  i.e. 
the  Y  process  does  not  admit  derivatives.  Sacks  and  Ylvisaker  (1968) 
considered  the  problem  of  selecting  a  T*e  Dr  so  that 

(1.3)  iKA'J)  *  inf  ^(A"1) 

TeD 

n 

or,  alternately 

■  SUP  *<*»> 

TeD 

n 


(1.4) 
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where  ijj  is  some  criterion  function  which  measures  the  size  of  A^,  (e.g.» 

the  trace  or  determinant  function).  A  design  which  satisfies  (1.3)  is 
termed  iJ>l-optimum  whereas  a  design  satisfying  (1.4)  is  termed  tj>2-optimum. 

As  optimal  designs  are  difficult  to  construct,  Sacks  and  Ylvisaker 
(1968)  instead  developed  approximate  solutions  to  the  optimal  design 
problems  (1.3)  and  (1.4)  which  entailed  the  use  of  design  sequences. 

The  method  they  used  for  constructing  such  sequences  was  to  choose  the 
elements  of  the  nth  design  to  be  the  n-tiles  of  a  continuous 
density  function,  h,  with  support  on  [0,1].  A  design  sequence  constructed 
in  this  manner  is  called  the  regular  sequence  generated  by  h.  This 


in  this  manner  is  called  the  regular  sequence  generated  by  h.  This 

relationship  is  abbreviated  (T^)  is  RS  (h) . 

*  .  .  .  - 
A  design  sequence,  {T  },is  said  to  be  asymptotically  ^1-optimum  if 

n 

(1.5)  limfinfiMA  “1)  -  'MA-1) )[  <MA~J)  -  ♦(A"1)f1  =  1 

n-+°°  TeD  n 

n 


and  is  asymptotically  \J/2-optimum  if 


(1.6)  lim  [i|»(A)  -  sup  iMAt)][iMA)  -  lHA^)  ]  *  1. 

n-*>°  TeD  n 

n 

Sacks  and  Ylvisaker  (1968)  derived  densities  having  the  property  that 
the  corresponding  regular  sequence  is  asymptotically  </>l  or  t)>2-optimum. 

Then  if  h*  is  such  a  density  and  (T*  }  is  RS  (h*),  their  approximate 

* 

solution  to  the  optimal  design  problem  is  T  . 

For  k  >  2  it  has  not  been  possible  to  characterize  the  asymptotic 

variance  behavior  or  obtain  asymptotically  optimal  design  sequences  for 

8^,.  Instead,  it  has  been  necessary  to  use  the  less  natural  observation 

set  Y  *  {Y{^  (t)  I  j«0,...,k-l,  teT)  and  the  corresponding  best  linear 
k,T 

unbiased  estimator  8.  _.  Hence,  for  k  >2,  most  of  the  available  literature 

k ,  T 

pertains  to  8.  rather  than  8  (c.f.  Sacks  and  Ylvisaker  (1970)  and 

X  f  JL  • 

Wahba  (1971,  1974)).  This  is  unfortunate  as  derivative  information  on 


the  Y  process  will  frequently  not  be  available. 
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In  this  paper  we  examine  the  asymptotic  behaviour  of  the  variance 
of  6t  where  k  is  allowed  to  be  any  fixed  finite  positive  integer.  Using  results 
pertaining  to  the  approximation  of  functions  by  splines  subject  to  boundary 
conditions  it  is  possible  to  extend  the  asymptotic  results  of  Eubank,  Smith 
and  Smith  (1981)  to  a  wider  class  of  processes  and  multiparameter  situations. 
The  class  of  processes  considered  includes  the  case  when  the  error  process 
is  a  multiple  integral  of  Brownian  motion  whose  importance  in  terms  of  (1.1) 
is  well  known  (c.f.  Sacks  and  Ylvisaker  (1970)). 

Our  results  stem  from  certain  asymptotic  properties  of  optimal 
quadrature  formula  derived  by  Barrow  and  Smith  (1979) .  The  implications 
of  their  work  to  the  approximation  of  functions  by  splines  under  boundary 
constraints  are  explored  in  Section  3.  These  results  are  related  to  the 
optimal  design  problem  in  Section  4.  As  an  illustration  of  the  relation¬ 
ship  between  these  two  problems  consider  the  instance  when  X(*)  is  a  (k-1)- 
fold  multiple  integral  of  Brownian  motion.  In  this  case  optimal  designs 


for  the  one  parameter  case  (J  =  1)  are  obtained  by  minimizing 

1  /  ...  n  3kR(s  t. )  \  2 

/  (f(k)(s>  -  I  a.  -3^ - )  ds 

0  V  i=0  1  3s*  / 


with  respect  to  (t  ,...,t  )  and  (a  ,  —  ,a  ).  It  will  be  seen  that 
O  n  0  n 

(kl_,  3kR(s,t) 

ts)  -  - j -  is  a  spline  of  order  k  with  a  knot  at  t  and  that  both 

3s 

and  will  satisfy 

f(k+j)(l)  =  r(k+j)(l)  =  0 

for  j  =  0,...,k  -  1  (except  for  t  =  1  when  j  =  k-1).  Consequently,  for 
this  type  of  process,  the  optimal  design  problem  is  equivalent  to  a 
variable  knot  spline  approximation  problem  where  both  the  function  and 


splines  must  satisfy  boundary  conditions. 


v  ' 
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We  present  asymptotic  results  in  Section  2  for  the  special  case 
of  J  =  1  with  proofs  in  Section  4.  In  Section  5  the  case  of  J  >  1 
is  examined.  A  theorem  is  given  which  allows  for  the  proof  of  results 
analogous  to  those  of  Sacks  and  Ylvisaker  (1970)  for  the  multiparameter 
setting  except  without  the  need  for  derivative  information. 

2.  The  case  of  one  parameter.  For  the  purpose  of  this  section 
we  will  restrict  our  attention  to  the  case  of  J  *  1.  Consequently, 
model  (1.1)  can  now  be  written  as 

(2.1)  Y(t)  -  8f  (t)  +  X(t)  ,  t  e  [0,1]. 


For  model  (2.1)  an  optimal  n+1  point  design  is  a  T*  e  which  satisfies 

A  A 

(2.2)  Var(B  )  =  inf  Var(8  ) 

T  TeD  T 


n 

and  an  asymptotically  optimal,  design  sequence,  {T*},  must  satisfy 

n 


(2.3) 


lim 
n-*  °° 


Var(B„,*)  -  Var (6) 
T* 
n 


inf  Var  (6  )-Var(B) 

TeD  T 

n 


1 


The  optimal  design  problem  can  now  be  formulated  as  a  nonlinear 
minimum  norm  approximation  problem  for  f.  Let  H (R)  denote  the  RKHS 
generated  by  the  covariance  kernel  R  which  is  isometrically  isomorphic 
to  the  Hilbert  space  spanned  by  the  Y  process  (c.f.  Parzen  (1961a,  1961b)) 
and  let  |  |  •  1 1  denote  the  norm  in  H (R) .  It  has  been  shown  by  Sacks  and 
Ylvisaker  (1966)  that 

(2'4)  Var(8T)  =  ||*Tf||R 
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where  J?T  denotes  the  H(R)  orthogonal  projector  for  the  subspace 

rt=  span{R(«,t) |t  eT}.  As  IIrt*IIr=  1 1 f I  I ^  -  I ! f  “  RTf||^  it  follows 

that  T*  is  an  optimal  design  if  and  only  if 


(2.5) 


If  -  Vfl 


inf 

TeD 


If  -  V' 


Consequently,  the  optimal  design  problem  is  equivalent  to  the  problem  of 

finding  (when  it  exists)  the  best  H(R)  approximation  to  f  from  the  nonlinear 

manifold  R  =  U  R^  .  Although  this  latter  problem  is,  in  general, 

TeD 

n 

quite  difficult  to  solve,  in  certain  instances,  H(R)  and  R  consist  respec- 

n 

tively  of  functions  and  splines  which  satisfy  certain  boundary  conditions. 

In  this  event  the  problem  becomes  amenable  to  analysis  as  will  be  shown  in 
Section  4. 

Now  and  in  further  discussions  we  will  consider  a  specific  class  of  Y 
processes.  Define  the  covariance  kernel 
1  ,k-l,  »k-l 

(2.6)  K(s,t)  :  =  / 

°  (k-1) : 

k  k 

where  x+  *  x  if  x  ^  0  and  is  zero  otherwise.  Denote  by  W(t)  the  zero 
mean,  normal  process  corresponding  to  K,  i.e.,  the  (k-l)-st  multiple 
integral  of  Brownian  motion,  and  let 

IW(t)  -  E[W(t)  |w(l)  (1)  ,  i  =  k-q, . . .  ,k-l] ,  1  £  q  1  k 

W(t)  q  *  0  . 

where  q  is  some  fixed  but  arbitrary  integer  between  0  and  k.  We 
now  take  R  to  be 


(2.8) 


R(s,t) :  *  Cov(Z (s) ,  Z(t) ) . 
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More  specifically,  let 

K(i,j) (s,t): 

Then,  for  q  1 

(2.9)  R(s,t)  =  K(s,t)  -  v(s)B_1v(t) 

where  v(s)  =  (K^0,Jc  ^  (s,l)  , . . . ,  K^0,5c  ^(s,l))'  and  the  ijth  element 
of  the  matrix  B  is  K^'^(l,l),  i,  j  =  k-q, . . . ,  k-1.  One  consequence 
of  this  choice  for  R  is  that  the  class  of  processes  to  be  considered 
includes  those  which  have  covariance  structures  like  that  of  (k-1) -fold 
integrated  Brownian  motion  (q  =  0)  and  (k-1) -fold  integrated  Brownian 
bridge  (q  =  1)  processes. 

We  now  state  two  theorems  regarding  the  behaviour  of  BT  for  which 
the  proofs  will  be  presented  in  Section  4. 

Theorem  2.1  Let  feH(R)  DC2k[  0,1] .  If  h  is  a  continuous  density 

on  [0,1]  and  (T  }  is  RS(h)  then 
n 

(2.10)  lim  n2k |  | f  -  R  f|  \l  =  Ck2 
n-»°° 

2  I B  I 

where  =  1  2k1 /2k!  and  B2k  is  the  2k -th  Bernoulli  number. 

In  view  of  (2.4)  and  (2.5)  it  is  clear  that  equation  (2.10) 
characterizes  the  asymptotic  behaviour  of  the  variance  of  6^  in  terms 
of  the  density  defining  the  designs.  The  next  theorem  concerns  the 

limiting  behaviour  of  inf  Var  (6_)  and  provides  a  density  which  generates 

TeD  T 
n 

an  asymptotically  optimal  design  sequence. 


i+j 

5  K  (s ,  t) 


3s1 at3 
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It  is  important  to  note  that  when  k  >  2  the  bound  inf  Var (B  )  may 

TeD  T 

n 

not  be  obtainable  without  the  use  of  derivative  information.  This  point 

will  arise  as  a  result  of  the  proof  of  Theorem  2.2  and  has  been  discussed 

by  Sacks  and  Ylvisaker  (1966).  However,  by  sampling  according  to  the 

density  (2.11)  an  approximate  solution,  Tn,  to  the  optimal  design  problem 

can  be  obtained  which,  for  large  n,  will  behave  like  an  optimal  design 

with  regards  to  the  corresponding  variance  of  the  BLUE.  It  is  also  of 

interest  to  note  that  the  limit  in  (2.12)  was  not  previously  known  to 

exist  although  certain  results  given  by  Sacks  and  Ylvisaker  (1970) 

2x 

had  the  consequence  of  bounding  lim  n  {Var(8T  )  -  Var (8)}  between  two 

n-*®  n 

numbers.  The  case  of  q  =  k  was  considered  in  Eubank,  Smith  and  Smith  (1978). 


Variable  knot  spline  approximation  with  boundary  conditions. 


this  section  we  develop  certain  mathematical  preliminaries  which  will  be 

X 

used  in  latter  sections.  For  TeD^  let  denote  the  linear  manifold  of  piece- 
wise  polynomials  of  order  k  (degree  <  k)  on  [0,1]  in  C  with  breakpoints  at 

lc 

ST  is  usually  called  the  set  of  splines  of  order  k  with  knots 
at  T.  Also  define  the  nonlir  >ar  manifold  of  all  splines  with  n  -  1 
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k  k 

distinct  knots  by  S  *  U  S„  .  Since  the  results  which  follow  involve 

n  „  „  T 
TeD 

2  n 

approximation  in  the  L  [0,1]  norm  we  adopt  the  notation 

| |f | | :  =  (/Qf2(t)dt)1/2  . 

2  2 
Given  f  e  L  [0,1]  and  a  fixed  set  of  knots,  T,  the  best  L  [  0,1] 

approximation  to  f  by  the  corresponding  splines  of  order  k  is  the 

k  k 

projection  of  f  onto  S^,  denoted  S^f .  if  the  knots  are  allowed  to  vary 

the  best  approximation  problem  becomes  nonlinear  as  one  is  attempting 

to  find  the  best  approximation  to  f  from  .  Thus,  in  this  latter  case, 

*  k 

one  is  attempting  to  find  s  e  S  so  that 

n  n 

(3.D  |  I  f~s*  |  |  -  inf  {  |  |f-s|  |  |s  e  Sk}  . 

s 

Finding  s*  in  (3.1)  is  a  variable  knot  spline  approximation  problem 
n 

and  is,  in  general,  quite  difficult  to  solve.  However,  when  f  e  C  [0,1], 
Barrow  and  Smith  (1978)  have  been  able  to  describe  the  asymptotic  behaviour 
of  | | f  -  s*|  |  and  suggest  a  scheme  for  knot  selection  which  is  asymptotically 
optimal . 

In  the  next  section  results  will  be  needed  regarding  variable  knot 
spline  approximation  when  both  f  and  s*  are  required  to  satisfy  certain 
boundary  conditions.  For  0  <_  q  <  k  let 

Ck,q[0,l]  =  (fe  Ck  [0,l]|fj(l)  =  0,  j  =  q,...,k-l) 

andaiven  TeD^  define  a  corresponding  spline  space 

Sk'*  -  {seSkjs^(l)  =  0,  j  =  q, ... .  ,k-l)  . 

1  T 

When  q=k  we  make  the  identification  Ck,k[0,l]  =  Ck[0,l]  and  Sk'k  *  Sk. 

A  variable  knot  spline  approximation  problem  in  this  setting  would  entail 

k  q 

finding  a  beat  approximation  to  f  from  U  S  .  As  it  is  not  clear  that 

TeD  T 
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such  a  best  approximation  exists  we  circumvent  this  difficulty  by  instead 
considering  the  best  approximation  to  f  from  the  closure  of  this  set 


S*'*  =  u  sk,cl  .  So  the  objective,  in  this  case,  is  to  find  s*  e  S 
**  _  _  t  q,n  n 

TeD  ^ 

n 

so  that 
(3.2) 


k,q 


f-s* 

1  q,n' 


=  inf { | | f-s  | 


S  E  S*'*}  . 

n 


The  spline  s*  may  have  knot  multiplicities  and,  consequently,  may  not  have 

q,n 

the  maximum  number  of  continuity  constraints. 

We  now  present  certain  results  regarding  the  asymptotic  behaviour  of 
s*  ^  and  S*'qf,  where  S^'q  is  the  orthogonal  projector  for  S^'q.  The  proofs 
of  the  following  theorems  are  a  consequence  of  results  pertaining  to  the 
asymptotic  properties  of  optimal  quadrature  formulas  given  by  Barrow  and 
Smith  (1979)  and  are  therefore  omitted. 


Theorem  3.1.  Let  f  e  Then, if  {T  }  is  RS(h) 


(3.3)  lim  nk|  !f-S*'qfl  |  =  C.  {/  ( ^  dx}1/2 


n-+°° 


where  C, 


”k‘'  .2k,  . 

n  0  h  (x) 

B  I  V/2 

2k'/2k!J  and  B ^  is  the  2k-th  Bernoulli  number. 


Theorem  3.1  characterizes  the  asymptotic  behaviour  of  the  error  in 
k,g 

approximating  f  from  ST  in  terms  of  the  density  defining  the  knot 
sequence  {T  }.  The  next  theorem  is  concerned  with  the  free  knot  case 

Ti 

and  provides  an  optimal  density  for  knot  selection. 


Theorem  3.2. 


(3.4)  h(x) 


Let  f  e  C 
1*00 


,k, 

(x) 


^  0,1]  and  define  the  density 
I 2/2k+l 


(k) 


(t) 


2/2k+l 

dt 
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Then. if  T  is  RS(h) 
'  n 


(3.5) 


lim  n 
n-H» 


If  -S 


kfq 


:  lim  n 
n-*» 


inf 

TeD 


If 


s*'qf! 


=  C, 


n 


€■ 


f(k) (x) 


2/2k+l  \ 2k+l/2 
I  dx 


) 


Theorem  3.2  has  the  implication  that  the  sequence  of  fixed  knot 

approximations  to  f,{ST'qf},  generated  by  h  in  (3.4)  works  as  well, 

asymptotically,  as  the  corresponding  sequence  of  best  free  knot  approxi- 

mations.  This  suggests  the  use  of  S^'4f  as  an  approximate  solution  to 

n 

problem  (3.2).  It  is  worthwhile  to  note  that  both  Theorems  3.1  and  3.2 
deal  exclusively  with  boundary  constraints  which  arise  as  a  result  of 
the  particular  design  problem  being  considered  and  are  readily  extended 
to  other  situations. 


4.  Proof  of  theorems.  In  this  section  we  explore  the  connection 
between  the  results  of  Section  3  and  the  optimal  design  problem 
discussed  in  Section  2.  In  particular , proof s  are  given  for  Theorems  2.1 
and  2.2. 

Fix  k  and  q  and  let  R  have  the  form  (2.8).  It  is  readily  shown  that 
(-1)  R(s,t)  is  the  Green's  function  for  the  boundary  value  problem 


(4.1) 


g(j)  (0)  =  0 
g(j)  (1)  =  0 


j  =  0, . . . ,k-l 

j  =  k-q, . . . ,k-l,k+q, . . .2k-l. 


Thus  it  follows  from  this  property  (or  directly  from  (2.9))  that  r  (s) :=R(s,t) 

when  considered  as  a  function  of  s  for  fixed  t  is  a  spline  of  order  2k  and 
2k- 2 

continuity  class  C  with  a  knot  at  t.  Consequently, Rn  is,  in  this  case, 


a  nonlinear  spline  manifold  and  the  optimal  design  problem  (2.5)  is  in 
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fact  a  variable  knot  spline  approximation  problem. 

The  RKHS  generated  by  R  is  seen  to  consist  of  functions  with  k-1 
absolutely  continuous  derivatives  and  is  given  by 

|{fjf(j) (0) =0  j=0,... ,k-l,f (j) (l)=0,j=k-q,...k-l,f(k)eL2[  0,1]},  i<q<k 
{f  |  f (j)  (0)=0,  j  =  0, . . .  ,k-l,  f(k)e  b2  [0,1]}  ,  q  =  0. 

The  inner  product  for  f  ang  g  in  H(R)  is  given  by 

(4.3)  <(f,g \  =  /  f (k) (x)g(k) (x)dx 

0 

2 

For  TeD  define  the  L  [0,1]  subspace 
n 

(4.4)  R*  :  »  span{r^k)|rt  e  R^}  . 

In  view  of  (4.3)  we  have 

(4.5)  IU-Hj.fl!,  -  llf<kl  -  (\f)k]  I  -  Ilf00  -  *k  (,k>ll 


k  2  k 

where  RT  is  the  L  [0,1]  orthogonal  projector  for  RT.  Consequently,  the 

optimal  design  problem  is  equivalent  to  finding  (when  it  exists)  the  best 

2  (k)  k 

L  [0,1]  approximation  to  f  from  the  splines  in  U  R  . 

TeD  T 
n 

To  prove  Theorems  2.1  and  2.2  we  will  show  that  the  optimal  design 

problem  is  nearly  the  equivalent  of  a  variable  knot  spline  approximation 
k ,  q 

problem  for  .  This  might  be  expected  as  it  is  apparent  from  the 

boundary  conditions  (4.1)  that  r^k^£  Sk,<*  for  t  e  t\{0,1},  i.e.,  for 

t  £  {t, ,...,t  ,}.  Hence,  we  are  in  a  situation  similar  to  that  consi- 

l  n-l 

dered  in  Section  3.  The  problem  preventing  the  immediate  proof  of  Theorems 
2.1  and  2.2  is  the  disparity  between  the  dimensions  and  elements  of  the 
sets  ST'y  and  R^,.  It  is  readily  seen  that  ST'q  has  dimension  n  +  q  -  1 
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Jc  (Jt  j  fc,q 

and  it  will  be  shown  subsequently  that  R^  has  dimension  n  with  ^  st* 

(for  q<k).  To  clarify  and  resolve  these  difficulties  it  will  be  helpful  to 

(k) 

further  analyze  the  properties  of  r  . 

Now  consider  the  form  of  R  given  in  (2.9).  For  q  >  0  let 

v(s)  =  (v  (s)  , . .. , v  (s) ) ' 
o  q~l 

where 


V.(s) 


K(0,k-q+i) 


(s,l) 


for  i  =  0,...,q-l.  Using  (2.6)  we  have  that 

(4-6)  3kK(s,t)  =  _Lt_-3)+“1..  =:K(k)(s) 

3sk  (k-1) !  t 


and 

(4.7) 


vf 1 1  (s) 


(1-E) 


q-i-i 


(q-i-l) ! 


(k) 

In  view  of  (4.3) ,  r  (s)  is  now  recognized  as  the  error  function  resulting 
2  (k) 

from  the  L  [0fl]  approximation  of  K  (s)  from  the  subspace 


P 

q 


=span{l. 


(1-s) 


g-s)*3"1, 

' (q-l)  1  1 


This  fact  has  several  consequences  which  are  of  interest.  First  it  is  seen 

k  i  2  (k) 

that  Rt-L  (in  L  norm).  It  also  follows  that,  for  q  <  k,  r^  (s)  is  a 

(2k-l)  (k 

polynomial  of  degree  k-1  which  satisfies  r^  (s)  =  1  and,  therefore,  r^ 

k  n  (k  1  (k ) 

is  not  contained  in  ST  •  When  q  =  k,  '  (s)  e  P^  so  r^(s)  will  vanish 

(k)  (k) 

identically.  Finally,  as  Kv  (s)  is  zero  on  [0,l],  r  must  vanish  as  well 

o  o 

k 

which  implies  that,  for  q  <  k,  the  dimension  of  R^  is  n  (the  vanishing 
(k) 

of  r  entails  that  observations  taken  at  zero  provide  no  information  about 
o 

$  a  fact  deduced  more  directly  from  (2.1)  and  (4.2)). 
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For  T  c  Dn  define  the  "design"  ?'  by  deleting  the  point  t^  =  1  from  T. 

For  designs  of  this  form  we  have  r£,  C  sk,q  .  Approximations  from  r£,  and  Sk,q 
are  related  by  the  following  lemma. 


Lemma. 


to  f 


(k) 


k  (k)  2 

If  f  e  H (R)  and  T  £  Dr  then  RT,f  is  the  best  L  [0,1]  approximation 

.  k,q 
from  ST 


Proof.  For  0  <  q  <.  k  the  dimension  of  R^,, 
in  L2  norm)  and  Rk,  C  Sk,q,  we  have 


is  n-1.  Since,  for  q  >  0, 


1  P 

q 


In  addition,  through  use  of  the  boundary  conditions  in  (4.2)  and  integration 

by  parts,  it  is  seen  that  if  g  £  H(R)  and  p  e  P^  then 
1 

g(k) (x)p (x) dx  -  0  . 

(Jr  )  Jr  (fc  ) 

As  f  -  RT,f  is  in  H (R)  it  follows  that  f  -  J^.f  is  orthogonal  to  Pg  and 

Rk, .  Consequently,  f  ^  is  orthogonal  to  Sk'q  for  0  <  q  <_k.  For 

k  k  0 

q  =  0  the  lemma  is  verified  through  noting  that  in  this  case  R^,  =  s'  . 


To  complete  the  proof  of  Theorems  2.1  and  2.2  the  difficulty  associated 
with  an  observation  taken  at  1  must  be  resolved.  This  is  accomplished  through 
noting  that 

(4.8)  ||f  -  ^tf||R  =  ||f(k)  -  s£'q  f(k,||  >  I  If  (k)  1 1  >  ||f(k)-Skf(k)|| 

Theorems  2.1  and  2.2  now  follow,  through  use  of  the  lemma  and  equation  (4.8), 

from  Theorems  3.1  and  3.2  respectively.  It  is  important  to  note  that  in  the 

free  knot  case  our  results  necessarily  pertain  to  the  best  H (R)  approximation 

to  f  from  the  closed  set  R  rather  than  R  and,  hence,  derivative  information 

n  n 

A 

may  be  required  to  obtain  the  bound  inf  Var(B  )  for  any  particular  n.  However, 

TED  T 
n 
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when  it  is  not  possible  to  sample  derivatives  of  the  Y  process  an 
approximate  solution  may  be  obtained  through  sampling  according  to  the 
density  (2.11). 


5.  The  case  of  many  parameters.  We  now  consider  the  general  case  of 
model  (1.1)  with  J  >  1.  Assume  that  i|/,  the  criterion  function,  is  a 

continuous  real-valued  function  on  the  non-negative  matrices  which  satisfies 
iMO)  =0  and  iMB)  i|>(C)  if  B-C  is  a  non-negative  matrix.  Given  a 

particular  criterion  i)/  we  wish  to  construct  asymptotically  4)1  or 

ii2  optimum  design  sequences.  To  do  so  it  will  suffice  to  prove  an 

analog  of  Sacks'  and  Ylvisaker's  (1968)  Theorem  3.2  for  covariance 

kernels  of  the  form  (2.8) 

2k 

Theorem  5.1,  Let  f^eC  ro,l)f|H(R),  j=l,...,J,  and  let  a^,...,aj  be 
a  set  of  positive  constants.  Then, for  anv  design  sequence  tTn^ 


(5.1)  lim  inf  n2kI  a  |Jf  -R  fj|*  >  C  ' 
n-*“  j-1  3  3  Tn  3  R 


(' 1  ,  J  ,  v  .l/2k+lv: 


l/2k+1^2k+l 


X  (say) . 


If  {T  }  is  RS (h)  where 
n 


(5.2) 


h(x) 


(X 


l/2k+l 


l/2k+l 


ds 


then 


_.  2k  ; 
lim  n  la. 

n-*»  j-1  3 


f  .-R_  f  .| 
j  T  t  1  ' 

J  n  J  R 


(5.3) 


*  X  . 
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Proof.  First  note  that  since  (5.3)  is  an  immediate  consequence  of 
Theorem  2.1  it  will  suffice  to  prove  (5.1).  The  latter  result  can  be 
obtained  through  modification  of  work  by  Barrow  and  Smith  (1978) .  We 
highlight  the  differences  here  and  refer  the  reader  to  their  paper  for 
more  details. 


Through  use  of  the  lemma  in  Section  4  and  the  fact  that,  for 
g  e  L2  [0,1]  ,  ||g-Sk,qg||  ||g-S^g||  it  is  seen  that  (5.1)  will  be 

If 

proven  when  it  can  be  shown  that  for  gi  £  C  [0,l],  j  =  1, _ ,J, 


..  v  II  0k  |  1 2  2.r  .  Z.  .  (k)  ,  .  ,2.  l/2k+l,2k+l 

(5.4)  E  a.  |  |g.-$  g.(  I  >C  (j  (  £  a  (g  <*>>  >  dx  > 

j=1  J  J  An  J  K  0  1=1  J  J 


The  proof  of  (5.4)  now  proceeds  in  a  step-wise  manner  similar  to  the 


proof  of  Theorem  2  of  Barrow  and  Smith  (1978).  The  inequality  (5.4)  is 


first  shown  to  hold  when  g. (t)  =  C.t  /k\  ,  where  C.  is  a  constant,  and 

i  j  i  i 

g.  e  Ck[0,l]  with  E  a.g.*k*(t)2  >_  5  >  0  before  finally  consi- 
dering  the  general  case  of  g i  e  C* [0,1] .  The  details  involved  in  the 


then  when 


vertification  of  each  of  these  cases  may  be  deduced  from  Barrow  and 
Smith  (1978). 


Theorem  5.1  provides  the  crucial  result  for  obtaining  asymptotically 
optimal  design  sequences  in  the  multi-parameter  case.  It  is  now  possible 
to  obtain  analogs  of  the  theorems  given  in  Section  4  of  Sacks  and  Ylvisaker 
(1968)  using  similar  methods  of  proof.  Examples  of  these  results  are 
provided  by  the  following  two  theorems  which  deal  with  the  trace  (tr) 
and  determinant  (det)  criteria. 

Theorem  5.2  Let  <J/(B)  *  tr(BM)  where  M  is  a  non-negative  J  x  J  matrix 

and  define  the  density 

<S.5)  h(x).  „  [»(x)’M»(x)11/2k‘>~1 

W *  rl  * l/2k+l 

J  (furM^t)]  dt 


where 


(5.6)  4>  (x)  : 

Then  the  design 
ij)  2 -optimum. 


(f|2k)(x),.  . 


.  ,  fj<2k)(x))V 


sequence  (T  }  which  is  RS(h)  is  asymptotically 
n 


Theorem  5.3.  Let  =  det(B)  and  define  the  density 


(5.7) 


h(x) : 


[*' (x)A_1^(t)]1/2k+1 

~1  "  l/2k+l 

/  t  4>  (t)  (t)  ]  dt 

0 


Then  the  design  sequence  {T^}  which  is  RS(h)  is  asymptotically 
i(j2-optimum. 
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